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This paper reports on the knowledge for teaching mathematics of 294 pre-service 
primary teachers from seven Australian universities participating in a project aimed 
at establishing a culture of evidence-based improvement of teacher education. The 
project was funded by the Australian Learning and Teaching Council. Rasch 
measurement techniques were used to validate and obtain performance measures on 
an overall Teacher Knowledge scale and three subscales (beliefs, content knowledge, 
and pedagogical content knowledge). The relative difficulties of items on each of the 
three subscales are discussed and differences between the participants' performances 
on each subscale and the overall scale according to level of education (prior to their 
pre-service teacher education course), previous mathematics study, course type, 
mode of study, and confidence to teach mathematics at the grade levels for which 
they were being prepared, are examined. The findings contribute to the 
establishment of an evidence-base for pre-service teacher education, and they also 
raise questions about the knowledge with which pre-service teachers leave teacher 
education, and current understandings of how important aspects of the knowledge 
teachers need can be measured. 

Teacher education in Australia has come under increasing scrutiny due to the 
emergence of a national, rather than State-based, education policy environment. 
This new environment features a national assessment program in literacy and 
numeracy (NAPLAN) for school students in Years 3, 5, 7, and 9 and a national 
curriculum in mathematics, English, science, and history, from Foundation to 
Year 10. In addition, national professional standards for teachers and a national 
system for accreditation of pre-service teacher education programs (Australian 
Institute for Teaching and School Leadership [AITSL], 2011a; 2011b) have 
replaced State-based frameworks. The AITSL Program Standards stipulate that 
teacher education providers must be able to demonstrate, at the time of initial 
program accreditation, that graduates will meet the Graduate career stage of the 
national professional standards. These latter developments appear to be 
underpinned by assumptions about the importance of quality in pre-service 
teacher education programs and the contribution of such programs to teacher 
quality. However, until recently there existed little research on the opportunities 
to learn provided by teacher education programs and their relative effectiveness 
(Tatto, Lerman, & Novotna, 2010). This paper arises from a study that aimed to 
provide an evidence-base for effective models of pre-service mathematics teacher 
education. It reports the outcomes of pre-service teacher education programs in 
terms of the knowledge for teaching mathematics demonstrated by a sample of 
prospective primary school teachers in seven Australian universities. 
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Factors Affecting the Outcomes of Teacher Education 

Internationally, there is increasing interest in the effectiveness of mathematics 
teacher preparation programs and the influence of program structure on 
prospective mathematics teachers' knowledge. Only a few years ago, Blomeke, 
Felbrich, Muller, Kaiser, and Lehmann (2008) noted that the few empirical 
studies that had been reported in this field tended to be small-scale, short-term, 
and conducted within the researchers' own institutions and with their own 
students. According to Blomeke et al., the main problem was that teacher 
education research "lacks a common theoretical basis, which prevents a 
convincing development of instruments and makes it difficult to connect the 
studies to each other" (p. 719). These authors tackled the theoretical problem by 
proposing a model for measuring effective teacher education in terms of the 
professional competence of future teachers. Professional competence was 
conceptualised as comprising professional knowledge, professional beliefs, and 
personal characteristics, with each of these components of competence 
acknowledged to have several inter-related dimensions. Blomeke et al. went on 
to propose factors that influence the development of professional competence, 
including individual characteristics of future teachers, institutional 
characteristics of teacher education, and the systemic features of teacher 
education within specific countries. The resulting model of professional 
competence and factors that influence its development provided the theoretical 
framework for an international comparative study of mathematics teacher 
education in six countries, "Mathematics Teaching for the Twenty-First Century 
(MT21)" (see Schmidt et al., 2008, for a report on findings). 

Developments since Blomeke et al.'s (2008) review include probably the best 
known study of prospective teachers: the Teacher Education and Development 
Study in Mathematics (TEDS-M). TEDS-M was an international quantitative 
comparative study of the preparation and competencies of primary and 
secondary mathematics teachers carried out in 15 countries from 2006-2009, 
aimed at understanding how national policies and institutional practices affect 
the outcomes of mathematics teacher education. Data were collected about the 
characteristics of prospective teachers as well as outcome measures of their 
professional competencies. The conceptual model of teacher professional 
competencies was similar to that proposed by Blomeke et al. (2008) in that it 
recognised both cognitive abilities (professional knowledge) and affective- 
motivational characteristics (beliefs, professional motivation, self-regulation) as 
key criteria for measuring effective teacher education (see Dohrmann, Kaiser, & 
Blomeke, 2012, for a discussion of the conceptualisation of mathematics 
competencies in the TEDS-M study). Information was also gathered about future 
teachers' opportunities to learn, defined in terms of the content and teaching 
methods experienced during teacher education, and relationships between 
opportunities to learn and knowledge outcomes were examined. The significant 
role of teacher knowledge within conceptual models of teacher professional 
competencies makes it important to develop clear definitions and well-justified 
measures of the knowledge required for mathematics teaching. 
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Professional Knowledge Required for Teaching 

It has long been recognised that teachers require more than knowledge of the 
subject matter for effective teaching. Shulman (1987) suggested seven categories 
of knowledge that might make up the knowledge base for the teaching 
profession: content knowledge, pedagogical knowledge, pedagogical content 
knowledge (PCK), curriculum knowledge, and knowledge of students, 
educational contexts, and the purposes of education. In mathematics education 
there has been much interest in the nature of PCK, which is concerned with the 
most useful ways of representing and formulating mathematics that make it 
comprehensible to others. Chick, Pham, and Baker (2006) used a framework 
developed from classroom observations to investigate the PCK of individual 
teachers. The framework describes aspects of PCK that are "clearly PCK" (e.g., 
knowledge of student ways of thinking about a mathematical concept), "content 
knowledge in a pedagogical context" (e.g., knowledge of the connections 
between mathematical topics), and "pedagogical knowledge in a content 
context" (e.g., knowledge of strategies for engaging students). The description of 
these categories highlights the interconnectedness of mathematics content 
knowledge (MCK) and PCK and the difficulty of distinguishing between them. 

In an effort to tease out the relationship between PCK and content 
knowledge, Deborah Ball and her colleagues have introduced the notion of 
Mathematical Knowledge for Teaching (MKT) (Ball & Bass, 2000; Ball, Lubienski, 
& Mewborn, 2001; Hill, Ball, & Schilling, 2008). They argue that MKT has two 
strands, comprising subject matter knowledge and PCK. These researchers 
conceptualise PCK as made up of knowledge of content and students, 
knowledge of content and teaching, and knowledge of curriculum. 

The TEDS-M study distinguished among MCK, mathematics pedagogical 
content knowledge (MPCK), and general pedagogical knowledge (Dohrmann et 
al., 2012), with only MCK and MPCK assessed in all countries that participated 
in TEDS-M. The MCK test categorised items according to the level of difficulty, 
ranging from novice (content typically taught at the grade level to be taught by 
the future teacher), to intermediate (one or two grades in advance of the grade 
level the future teacher will teach) and advanced (three or more years beyond the 
grade level the future teacher will teach). The MCK test was designed to align 
with the TIMSS assessment framework, and so it addressed the cognitive 
domains of knowing, applying, and reasoning together with (for future primary 
school teachers) the content domains of number, geometry, algebra, and data. 
Even so, it was a challenging task for the TEDS-M researchers to reach agreement 
about what constituted MCK for future teachers, since different mathematical 
content domains receive different emphases in countries around the world. 

Conceptualising MPCK in the TEDS-M study proved to be still more 
difficult because of the different theoretical, educational, and cultural traditions 
of the participating countries. Nevertheless, two sub-domains of MPCK were 
agreed upon: curricular knowledge and knowledge of planning for mathematics 
teaching and learning; and knowledge of enacting mathematics for teaching and 
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learning (Dohrmann et al v 2012). The first sub-domain refers to more than 
knowledge of the mathematics curriculum, and includes "the ability to identify 
the key ideas in learning programmes, seeing connections within the curriculum, 
establishing appropriate learning goals and knowing different assessment 
formats" (p. 329), as well as selecting teaching approaches and anticipating 
student responses. The second sub-domain refers to the ability to analyse and 
diagnose student thinking and to interpret and evaluate their mathematical 
solutions and arguments, as well as to explain concepts and guide classroom 
discourse. 

Although much of the research on the knowledge required for effective 
teaching has concentrated on describing and categorising that knowledge, there 
is evidence that MCK and PCK work together in developing professional 
competence. For example, in a study conducted in Germany with a 
representative sample of Grade 10 classes and their mathematics teachers, 
Baumert et al. (2010) found that the level of teachers' mathematics PCK was a 
significant predictor of students' mathematical achievement whereas teachers' 
levels of MCK were not. However, these authors argued that both forms of 
knowledge are essential, since content knowledge forms the basis for 
development of pedagogically-oriented knowledge for teaching mathematics. 

Professional Beliefs 

The TEDS-M study framed teacher professional competencies in terms of both 
cognitive and affective characteristics, the latter including beliefs about 
mathematics and beliefs about the teaching and learning of mathematics 
(Dohrmann et al., 2012). In this study, beliefs about mathematics were held to be 
important for guiding the application of professional knowledge in practice. 
Beliefs of future primary teachers were investigated using an instrument 
comprising 12 Likert-style items that distinguished between a dynamic 
perspective (Mathematics as a process of enquiry) and a static perspective 
(Mathematics as a set of rules and procedures) (see Felbrich, Kaiser, & Schmotz, 
2012). The cultural orientation of the participating countries—either 
individualistic or collectivistic—was also measured using a separately validated 
instrument. Although there were strong variations in belief-perspectives within 
countries, between-country differences were also apparent and these seemed to 
be related to the individualistic-collectivistic orientation of a country. 

Rather than treating beliefs as a separate component of teachers' 
professional competencies, Beswick, Callingham, and Watson (2012) argued that 
beliefs should be included in any conception of teacher knowledge because these 
constructs are so closely intertwined in the context of practice. They developed a 
written profile to measure the beliefs, content knowledge, and PCK in 
mathematics of a sample of middle school teachers. A Rasch analysis 
demonstrated that this profile measured a single underlying construct, thus 
validating their holistic conception of knowledge for mathematics teaching. 


74 


Kim Beswick & Merrilyn Goos 


How do Opportunities to Learn Influence Knowledge Outcomes in 
Teacher Education? 

The TEDS-M study revealed great variation across countries and programs in 
opportunities to learn tertiary-level mathematics (Tatto & Senk, 2011). 
Knowledge outcomes also differed significantly between participating countries 
and between teacher education programs within countries (Blomeke, Suhl, 
Kaiser, & Dohrmann, 2012). For example, Singapore primary teachers 
participating in the TEDS-M study ranked first or second in MCK and MPCK, 
depending on pre-service program structure. However, it was not possible to 
establish connections between these strong knowledge outcomes and measures 
of opportunities to learn used in the TEDS-M study (self-reported opportunities 
to learn tertiary level mathematics, school level mathematics, and mathematics 
pedagogy within NIE's teacher education program) (Wong, Boey, Lim-Teo, & 
Dindyal, 2012). In contrast to the Singapore findings, the United States TEDS-M 
researchers found clear evidence of a relationship between opportunities to learn 
both MCK and MPCK (Schmidt, Houang, & Cogan, 2012). It may be that this 
relationship is more important in countries like the US that have highly 
decentralised education systems and where there is substantial within-country 
variation in the regulation of teacher preparation. 

The study reported here took into account research summarised above in 
developing a survey for measuring pre-service primary school teachers' 
knowledge for mathematics teaching. The survey included items concerned with 
MCK, PCK, and mathematical beliefs as outcome measures. Previous research 
confirms that teacher knowledge, and especially PCK, is difficult to measure. 
Multiple choice items are effective for use with large samples, but, as Chick 
(2011) noted, responses are elicited out of context and they do not give detailed 
information about the reasons for the choices made. Lesson observations and 
interviews are better suited to a deeper examination of the different aspects of 
PCK as they are manifested in practice, but these methods are time consuming 
and less effective for larger samples of teachers. While acknowledging the 
limitations of surveys for investigating PCK, the present study makes a 
contribution to the evidence-base informing pre-service teacher education and 
suggests further questions concerning the operationalisation and measurement 
of knowledge for teaching. 


The Study 

The 2-year Australian Learning and Teaching Council (ALTC) funded project. 
Building the Culture of Evidence-based Practice in Teacher Preparation for 
Mathematics Teaching (CEMENT), involved seven universities representing all 
states and the Northern Territory and providing a diverse range of teacher 
education programs. The project's aims, as reported by Callingham et al. (2011, 
pp. 901-902), were to provide: 

1. Evidence-based changes to mathematics education within the 
participating universities; 
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2. Recommendations about effective models of teacher education for 
teaching mathematics; 

3. Processes for bringing about change at unit and course level; and 

4. Progress towards a national culture of evidence-based practice in 
relation to mathematics teacher education. 

The study adapted its conceptual framework from that used in the TEDS-M 
study (Tatto et al., 2008). The framework identifies three domains assumed to 
comprise factors affecting outcomes of teacher education programs. The first 
domain is concerned with the characteristics of future teachers and includes 
background information such as age, gender, location, qualifications, study 
experiences, attitudes to and beliefs about mathematics and mathematics 
learning. The second domain considers the characteristics of teacher educators, 
including classroom teaching experience, qualifications, and beliefs about 
mathematics, mathematics teaching and learning, and the needs and capacities 
of the pre-service teachers with whom they work. The third domain 
encompasses characteristics of teacher education programs —mode and level of 
delivery (e.g., internal, external, or mixed mode; undergraduate vs. 
postgraduate); the balance between subjects teaching mathematics content, 
mathematics pedagogy, and general pedagogy; and the duration and 
organisation of practicum sessions. The outcomes of teacher education programs 
are conceptualised as beginning teachers' beliefs and knowledge relevant to 
teaching mathematics. 

The research reported here investigated different aspects of prospective 
primary school teachers' knowledge for teaching mathematics, and compared 
these outcome measures for groups of participants based on a range of future 
teacher characteristics and teacher education program characteristics. It provides 
starting points for the participating universities to examine their mathematics 
education practices and directions for further research into the relative 
effectiveness of a range of teacher education models and emphases in 
mathematics education. It thus contributes to aims 1, 2, and 4 of the project. 

Instrument 

An online multiple-choice survey was used to obtain data in a cost-effective, 
readily scored way. The 84-item instrument comprised 9 beliefs statements and 1 
confidence item with which respondents indicated the extent of their agreement 
on 5-point Likert-type scales; 45 items concerned with MCK; and 29 designed to 
address PCK. The MCK and PCK item numbers include the multiple parts of 
several questions. Each pre-service teacher who undertook the survey was 
offered all of the beliefs and confidence items along with a randomly generated 
set of 10 MCK and 11 PCK questions, some of which included multiple parts 
making the numbers of items responded to greater than 10 and 11 respectively. 
This design enabled the approximate time to complete the survey to be kept to 
45 minutes while allowing a greater number of items to be used and evaluated. 
The mean numbers of MCK and PCK items responded to by each participant 
were 22 and 19 respectively, with each MCK item answered by an average of 97 
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participants, and each PCK item answered, on average, by 119 participants. 
Qualtrics survey software (www.qualtrics.com) was used to deliver the survey to 
participants using a unique web-link. 

The process of survey development, described in detail by Beswick and 
Callingham (2011a), was collaborative. It drew upon the extensive combined 
expertise of the nine mathematics educators involved in the project, several of 
whom had published work related to the nature and development of beliefs and 
knowledge for teaching mathematics (e.g., Beswick et al., 2012; Chick et al., 2006). 
The nine beliefs items comprised three each related to the nature of mathematics, 
mathematics teaching, and mathematics learning and were modified from 
existing sources (e.g., Thompson, 1984; Van Zoest, Jones, & Thornton, 1994). 

The MCK items reflected content in each of the three content strands of the 
Australian Curriculum: Mathematics (Australian Curriculum Assessment and 
Reporting Authority [ACARA], 2012), namely Number and Algebra, Statistics 
and Probability, and Geometry and Measurement. The numbers of items 
designed to address each strand broadly reflected the amount of content in each 
strand in the curriculum and hence approximately half of the MCK items 
addressed Number and Algebra and one quarter covered each of the other two 
strands. Care was taken to include approximately even coverage of curriculum 
content from Foundation (the first year of schooling) to Year 7 (the first year of 
secondary school in most Australian states and territories). 

The PCK items were intended to capture the following aspects of the 
construct: (1) analysing/anticipating/diagnosing student thinking, (2) con¬ 
structing/choosing tasks/tools for teaching, (3) knowledge of representations, 
and (4) explaining mathematical concepts. These aspects of PCK are consistent 
with and further explained by the sub-categories of Chick et al.'s (2006) 
framework concerned with; student thinking and misconceptions, understanding 
the cognitive demands of tasks and the use of resources to support teaching, 
knowledge of using representations to model or illustrate a concept, and 
knowledge of strategies for teaching a concept. They also align with the TEDS-M 
sub-domains of curricular knowledge and knowledge of planning (aspects 2 and 
3) and knowledge of enacting mathematics for teaching and learning (aspects 1 
and 4) (Dohrmann et al., 2012). Typical items were presented in the context of a 
briefly described scenario with possible teacher responses from which par¬ 
ticipants could choose. 

Examples of items are shown in Table 1. The MCK items and many of the 
PCK items were scored right or wrong. Some PCK items, however, did not lend 
themselves to dichotomous scoring. The PCK item shown in Table 1, for 
example, was scored A = 0, B = 2, C = 2, D = 1 because B and C were both seen 
as helpful responses in terms of developing understanding. D represented a 
sophisticated approach that could be meaningful for students with appropriate 
prior knowledge but judged less likely to be helpful than B or C. 
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Participants 

Cohorts of primary pre-service teachers in each of the seven participating 
universities were invited to complete the survey. Data were received from 294 
pre-service teachers. Because the participants were volunteers and data about the 
relevant complete populations is not available, it is impossible to establish the 
extent to which the sample is representative of Australian pre-service teachers. 
The following details, however, provide a picture of the characteristics of the 
participants. More than three-quarters (n = 228, 77.6%) were full-time students 
and just under one-quarter (n = 66, 22.4%) were part-time. Approximately equal 
numbers were studying on-campus (n = 129, 43.9%) and off-campus (n = 131, 
44.6%), with the remainder (n = 33, 11.2%) undertaking a mix of on- and off- 
campus study. Most were planning to graduate in 2011 (n = 80, 27.2%) or 2012 
(n = 79, 26.9%). A further 37% were aiming to graduate in 2013 (n = 58,19.7%) or 
2014 (n = 51, 17.3%). Smaller numbers were aiming for graduation in 2015 (n = 
13,4.4%) or later (n = 12,4.1%). Most (n - 132,44.9%) were enrolled in a Bachelor 
of Education (BEd) (a 4-year course), one third (n = 98, 33.3%) in a combined 
degree program (at least 4 years), 46 (15.6%) were undertaking a Master of 
Teaching (MTeach) degree (2 years), and 17 (5.8%) were engaged in a Diploma of 
Education (DipEd) (1 year). 

The highest educational level prior to enrolment in their current course was 
secondary for 133 (45.2%) respondents. Sixty-five respondents (22.1%) held a 
bachelors degree and 39 (13.3%) and 33 (11.2%) respectively held advanced 
diplomas or certificates. Twelve (4.1%) had a post-graduate degree and a further 
10 (3.4%) a graduate diploma. One reported primary education as his/her 
highest prior educational level. 

Because of the range of educational jurisdictions in which the universities 
were located and the consequent variety of subject names. Year 11 and 12 
mathematics study options were worded as follows: Year 11/12 Mathematics or 
statistics subject that didn't count towards university entrance; Year 11/12 
general mathematics or statistics course that counted for university entrance; and 
Year 11/12 specialist mathematics or statistics intended for mathematics study at 
university. For brevity these options are referred to in this paper as non-pre- 
tertiary, pre-tertiary, and specialist. Two thirds (n = 196,66.6%) of the participants 
described the highest level of mathematics or statistics studied prior to their 
current course as Year 12. Of these the majority (n = 140, 47.6% of the total 
number of participants) had studied pre-tertiary mathematics. Forty-three 
(14.6%) had studied non-pre-tertiary mathematics and 13 (4.4%) had studied 
specialist mathematics. Forty-four (15.0%) reported their highest level of 
mathematics as Year 10 and a further 40 (13.6%) had studied some mathematics 
or statistics as part of a bachelors degree. Two (0.7%) reported studying 
mathematics at a Technical and Further Education (TAFE) college and 12 (4.1%) 
indicated that they could not remember the highest level of mathematics or 
statistics studied. None had studied mathematics at post-graduate level. 

The responses to the confidence item (shown in Table 1) were used as an 
additional background variable. Most participants indicated that they were fairly 
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confident (n = 159, 56.2%) to teach at the grade levels that they would be 
qualified to teach, with a further 38 (13.4%) describing themselves as completely 
confident. Most of the remainder (n = 58, 20.5%) indicated they were "A little 
confident" and 14 (4.9%) chose each of "Don't know", and "Not at all confident". 


Table 1 

Examples of items 


Beliefs 


Confidence 


Mathematics 

Content 

Knowledge 

(MCK) 


Pedagogical 

Content 

Knowledge 

(PCK) 


Periods of uncertainty and confusion are important for 
mathematics learning. 

Strongly Disagree Disagree Neither Agree nor Disagree Agree 
Strongly Agree 

Please rate your confidence to teach mathematics at the grade 
levels that you will be qualified to teach on the following scale. 

Not at all confident A little confident Don’t know Fairly confident 
Completely confident 

Steve buys a shirt that is discounted by 10% on the ticket. A sign on the 
rack stated, 'Discounted by a further 15%'. This is the same as a 
discount of what percentage off the original price of the shirt? 

A) 12.5% B)15% C) 17.5% 

D) 23.5% E) 25% 


A teacher sets the following proportional reasoning task for an upper 
primary class: 

Bill and Ben were out on a Sunday morning bike ride. After three 
quarters of an hour they passed a sign that showed they had ridden 
15 kilometres since they left home and that they still had 25 kilometres 
to reach their destination. How long will it take them to get there? 
Which of the following representations is most helpful for the teacher 
to develop the students' understanding of proportional reasoning in 
solving this problem? 


A) Cross multiplying 


Time (hr) 

Distance (km) 

3 / 4 \ 

/ 15 

x / 

< \ 25 


C) Ratio table 

Time 3 /4 hr V 4 hr lhr 

Distance 15km 5km 1km 25m 


B) Double line number 

ll i hr 3/ 4 hr lMhr 

-i-1-!-!-i— 

5km 15km 20km 25km 30km 


D) Find the unit rate: 
Riding 15 km in 3/ 4 hr is 
equivalent to riding 
1 km in 3 /4 5 -15 hr. 
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Data Analysis 

Rasch measurement models use the interaction between persons and items to 
position each item and person against an underlying construct on the same 
genuine interval scale. The scale units are logits, the logarithm of the odds of 
success (Bond & Fox, 2007). Because the items that comprised the instrument 
used in this study had differing structures the specific model used was Masters' 
(1982) Partial Credit Model. Rasch models are based on three assumptions. These 
are, (1) that the items work together to measure an underlying construct, (2) the 
underlying construct is measureable such that higher item scores are indicative 
of more of that construct, and (3) that items contributing to the construct are 
independent of one another (Bond & Fox, 2007). The initial step in using Rasch 
measurement is, therefore, to establish the extent to which these assumptions 
hold. 

To this end, Rasch analyses of all 84 items and then each of the three sets of 
items (Beliefs, MCK, and PCK) were conducted using Winsteps (Linacre, 2011) 
with a view to establishing scales in relation to which relative item difficulties 
and person performance measures could be calculated. In each case infit mean 
square values for both items (INMSQI) and persons (INMSQP), available from 
the Winsteps output, were used to evaluate the fit to the Rasch model (Bond & 
Fox, 2007). Fit values between 0.77 and 1.3 logits are generally acceptable with 1.0 
representing the ideal value (Keeves & Alagumalai, 1999). Winsteps also 
provides person and item reliability indices and separation statistics. Low person 
separation (<2) and person reliability less than 0.8 indicate uncertainty about the 
ordering of the ability measures of the respondents (Linacre, 2011). Similarly, low 
item separation (<3) and item reliability less than 0.9 indicate uncertainty about 
the ordering of the items in relation to their difficulty (Bond & Fox, 2007). Person 
reliability is enhanced by lengthening the test (i.e. having participants respond to 
more items) and increasing the range of abilities represented by the respondents. 
Item reliability can be improved by increasing the number of respondents 
(Linacre, 2011). Measures of the performance of each pre-service teacher were 
obtained for each of the overall survey and the Beliefs, MCK, and PCK subscales. 
These measures were used to compare groups based on responses to the 
confidence item, course type (e.g., BEd, MTeach), university, full or part time 
enrolment, mode of study, level of mathematics studied, prior educational level, 
and anticipated graduation year, using t-tests and analyses of variance. 

Results and Discussion 

Eighty of the 84 items formed a scale with satisfactory fit values for both items 
and persons (INMSQj = 0.93, INMSQ P = 0.97), suggesting that these items 
provided a measure of an underlying construct that we called Teacher 
Knowledge. The Cronbach a reliability statistic was 0.89. Person separation 
(0.75) and person reliability (0.36) were low, whereas item separation (12.57) and 
item reliability were high (0.99). This means that the precise person ability 
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ordering is uncertain. Nevertheless, the acceptable Cronbach (X suggests that the 
survey data provide a sound basis for looking for between group differences in 
Teacher Knowledge. Because the focus of the study was on providing evidence 
about the effectiveness of pre-service teacher education, the item reliability and 
the consequent confidence in the order of item difficulty, is of greatest 
importance. The nine beliefs items showed excellent fit to the Rasch model 
(INMSQj = 0.99, INMSQp = 1.01) and acceptable reliability (a = 0.79). Again, the 
person separation (0.60) and person reliability (0.27) were low but the item 
separation (13.47) and reliability (0.99) were high. All but one of the 45 MCK 
items comprised a scale that fitted to the Rasch model (INMSQj = 0.99, INMSQ P 
= 0.97). Cronbach a was 0.65. As for the overall Teacher Knowledge scale and 
Beliefs scale, the person separation (1.06) and person reliability (0.58) were low, 
whereas the item separation (3.54) and item reliability (0.93) were high. Finally, 
28 of the 29 PCK items also comprised a scale with excellent fit to the model 
(INMSQj = 0.99, INMSQp = 1.00) and Cronbach a of 0.58. The person separation 
(0.43) and person reliability (0.15) were low although the item separation (9.06) 
and item reliability (0.99) were high. The lower Cronbach a and person reliability 
measures for MCK and PCK scales reflect the high apparent levels of missing 
data resulting from the random assignment of subsets of each of these item types 
to survey respondents and the effective shortening of these parts of the survey 
that this entailed. In this context, the reliability statistics are acceptable. 

Relative Difficulty of the Different Scales 

Figure 1 shows box plots of the distribution of pre-service teachers' performance 
on each of the three scales. Beliefs, MCK and PCK. As was the case for pilot data 
reported by Callingham et al. (2011), the median performance declined across the 
scales. Flowever, in contrast to the pilot data, performance on the Beliefs scale 
was lower and there is greater overlap between the performance of the pre¬ 
service teachers in relation to MCK and Beliefs, and between MCK and PCK. 
Nevertheless, pre-service teachers found endorsing belief statements and, to a 
lesser extent, correctly answering mathematical content questions less difficult 
than responding appropriately to the PCK items. Items from each scale that 
proved most and least difficult are discussed in the following sections. 

Beliefs. The easiest beliefs statement to endorse (that is, the statement with 
which participants were most likely to agree) both related to mathematics 
teaching. They were; "Teachers must be able to represent mathematical ideas in 
a variety of ways" and "The teacher must be receptive to the students' 
suggestions and ideas". The most difficult statements to endorse were; 
"Acknowledging multiple ways of mathematical thinking may confuse 
children" and "The procedures and methods used in mathematics guarantee 
right answers". Apparently the pre-service teachers' openness to multiple 
representations and student contributions was accompanied by resistance to the 
notion that there are "right" answers in mathematics. Beswick and Callingham 
(2011b) reported a similar result and referred to Brown, McNamara, Flanley, and 
Jones' (1999) finding that pre-service teachers are influenced by their own 
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experiences of learning mathematics that commonly include fear of getting 
incorrect answers, to explain a similar finding. They speculated that mathematics 
teacher education helped reduce pre-service teachers' fear of failure, and that the 
associated emotional relief contributed to the attractiveness of statements that 
suggest multiple acceptable possibilities even if the implications had not been 
thought through. 



Figure 1. Distribution of pre-service teachers' performance measures on three 
subscales with outliers shown 

Two items relating to the nature of mathematics, "Mathematics is a beautiful and 
creative human endeavour" and "Mathematical ideas exist independently of 
human ability to discover them", were the next most difficult to endorse, and 
equally so. Again this mirrors the finding of Beswick and Callingham (2011b) 
and suggests that the nature of mathematics may receive little attention in 
teacher education courses. 

Mathematical content knowledge (MCK). The easiest items involved identifying 
pictures of a square, rectangle, non-rectangular parallelogram, and a non-square 
rhombus as parallelograms. In each case more than 96% of respondents 
answered correctly. The most difficult items included one requiring respondents 
to decide whether a parallelogram "has diagonals that cross at right angles when 
adjacent angles are different" is always, sometimes or never true (25.0% correct). 
The easy items involved shape recognition without reference to properties and 
thus required geometric thinking at Van Hiele's Visual level at which the 
appearance, rather than properties, of shapes are the basis on which they are 
recognised (Siemon et al., 2011). The most difficult geometry item, however, 
required abstract/relational geometric thinking (Siemon et al., 2011) in order to 
analyse the relationship between properties of a parallelogram. 


















82 


Kim Beswick & Merrilyn Goos 


Two items related to generalisations expressed algebraically were also 
among the five most difficult. These involved deciding whether a - b = b - a 
always, sometimes, or never (25.3% answered correctly); and whether a 2 < a, is 
always, sometimes, or never true (28.9% correct). Algebra requires a greater level 
of abstract thinking than does arithmetic (Malisani & Spagnolo, 2009) and hence 
the difficulty of these items is not surprising. Another item among the five most 
difficult required respondents to decide whether the product of an odd and an 
even number is sometimes, always, or never odd (30.9% correct). The need to 
know the meaning of "product" may account for many of the incorrect 
responses. The inclusion in this group of an item that asked pre-service teachers 
to determine the length of a line marked above a broken ruler with zero not 
aligned with the end of the line (25.5% correct) is more surprising. 

Overall the results for MCK suggest that although teacher education needs 
to attend to sophisticated concepts involving abstract thinking, the weak 
mathematics knowledge of entering pre-service primary teachers (Mays, 2005) 
means that simple knowledge and skills, such as the correct use of a ruler, cannot 
be taken for granted. We are also aware that subtle variations in wording can 
influence the interpretation of items. For example, in the MCK item shown in 
Table 1 the percentage discount asked for could be interpreted as just the 15% of 
the 90% of the original price (and hence 13.5% of the original price) rather than 
the total discount off the original price (and hence 23.5%). Since 13.5% was not 
an option respondents who arrived at that conclusion may have been prompted 
to realise that 23.5% was the desired response but it is possible that some of these 
respondents were disadvantaged by the omission of 13.5% as an option or 
wording that clearly indicated that it was the total discount that was meant. 

Pedagogical content knowledge (PCK). The easiest PCK item was shown in 
Table 1 and involved proportional reasoning. Most (84.9%) pre-service teachers 
responded with one or other of the responses that were scored 2. Only 4.4% chose 
the response scored 0. The transition from purely additive thinking to being able 
to reason proportionally when appropriate, is recognised as a key milestone in 
students' mathematical development (Sowder et al., 1998). This recognition may 
have led to the teaching of proportional reasoning receiving relatively more 
attention than other topics in teacher education programs. Alternatively, the 
result could be an artifact of the scoring of the options provided: only one was 
scored '0' and this may have been readily recognised as a meaningless procedure 
based on pre-service teachers' recollections of learning mathematics. The next 
easiest item involved a teacher's response to a scenario in which student 
calculators yielded differing answers to a calculation involving addition and 
multiplication. In this case 58.5% of responses scored 2 and a further 39% were 
scored 1. 

In contrast, just 12.2% of the pre-service teachers provided an appropriate 
response to a student's representation of the sum of V 4 and V 4 that suggested a 
total of 2 / 8 . This was the most difficult item. Chick (2011) described the 
controversy this item generated among the researchers to illustrate the inherent 
subjective judgements involved in devising PCK items: certain approaches are 
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privileged above others when, in fact, more than one teaching response could be 
seen as optimal depending upon the precise circumstances. The difficulty of 
efficiently measuring PCK, particularly aspects of it that depend upon teaching 
responses to the minute-by-minute unfolding of classroom events, seems 
unlikely to be easily overcome. 

Between Groups Analyses 

Comparisons of performance on the Teacher Knowledge scale and each of 
Beliefs, MCK and PCK were made between groups identified by responses to the 
confidence item, course type, university, full or part- time enrolment, mode of 
study, level of mathematics studied, prior educational level, and anticipated 
graduation year. Analyses of variance and t-tests were used to identify 
significant differences. No differences for any of the scales were found in relation 
to university, full or part-time enrolment, or anticipated graduation year. In 
addition, none of the groups considered differed in relation to either Beliefs or 
PCK. The significant differences that were found are described in relation to each 
background variable in the following sections. 

Differences according to confidence level. For overall Teacher Knowledge there 
were differences in performance between pre-service teachers who were "Not at 
all confident" (M = -6.65) or "A little confident" (M = -6.48), and those who were 
"Fairly confident" (M = -6.12), or "Completely confident", (M = -5.93), 
F(4, 278) = 10.84, p < .01. Those who chose "Don't know" (M = -6.40) to describe 
their level of confidence performed less well than those who were "Completely 
confident" (p < .05). The associations between confidence and Teacher 
Knowledge may reflect the pre-service teachers' awareness of either the extent of 
their mathematical or other knowledge or at least of their capacity to learn about 
sophisticated ideas. 

Pre-service teachers who were "Not at all confident" (M = -0.15) performed 
less well in terms of MCK than those who were "Fairly confident" (M = 0.60), or 
"Completely confident", (M = 0.64), F(4, 278) = 4.54, p < .05. Those who were "A 
little confident" (M = 0.14) demonstrated less MCK than those who were "Fairly 
confident", p < .05. As for Teacher Knowledge, awareness of one's level of MCK 
is likely to affect one's confidence to teach mathematics and so the association 
between MCK and confidence is unsurprising. 

Differences according to course type. On Teacher Knowledge those enrolled in 
a Diploma of Education (M = -5.78) performed better than those enrolled in 
either a combined degree (M = -6.25) or BEd (M = -6.24), F(3, 289) = 4.09, p < .01. 
Differences were found between the same groups for MCK, with DipEd pre¬ 
service teachers (M = 1.06) demonstrating greater MCK than those undertaking 
a combined degree (M = 0.30), F(3, 289) = 3.99, p < 0.01, or a BEd (M = 0.42), p < 
.05. These differences, and particularly those for in MCK, are not easy to explain 
because the sample comprised primary pre-service teachers: those studying for a 
DipEd would not necessarily have studied mathematics or statistics as part of a 
prior degree. 
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Differences according to mode of study. Differences related only to MCK with 
pre-service teachers studying in mixed mode (M = 0.04) performing less well 
than those studying off-campus (M = 0.55), F(2, 290) = 4.13, p < .05. There is no 
apparent explanation for this association, however, because relatively few of the 
participants were studying in mixed mode (n = 33, 11.2%) and so the result 
warrants caution. Possibly of greater significance is the lack of difference 
between on- and off-campus study modes. 

Differences according to highest level of mathematics or statistics completed. Pre¬ 
service teachers who had studied mathematics or statistics as part of a bachelor 
degree (M = -5.97) had greater Teacher Knowledge than those whose highest 
mathematics level was Year 12 non-pre-tertiary (M = -6.38), F(6, 287) = 3.05, 
p < .05. 

In terms of MCK, pre-service teachers who reported their highest 
mathematics level as Year 10 (M = 0.14) did less well than those who had 
completed Year 12 specialist mathematics (M = 1.06), mathematics at TAFE (M = 
2.16), or bachelor degree level mathematics (M = 0.79), F( 6, 287) = 5.74, p < .05. 
Completion of non-pre-tertiary mathematics at Year 12 (M = 0.10) was associated 
with lower MCK performance than Year 12 specialist mathematics (M = 1.06), 
mathematics at TAFE (M = 2.16), F(6,287) = 5.74, p < .05, or bachelor degree level 
mathematics (M = 0.79), F(6, 287) = 5.74, p < .01. Unsurprisingly, greater MCK 
was associated with having studied more and higher level mathematics. 

Differences according to highest prior education level. Prior educational level 
impacted on Teacher Knowledge performance, with those who had completed a 
bachelor degree (M = -6.03) performing better than those whose highest 
educational level was secondary school (M = -6.28), F(5, 286) = 2.25, p < .01. 

Differences were also found for MCK, with pre-service teachers with 
bachelor (M = 0.69) or post-graduate degrees (M = 1.01) demonstrating greater 
MCK than those whose highest educational level was either certificate (M = 0.26) 
or secondary level (M = 0.35), F(5, 286) = 2.49, p < .05. Again, because the 
participants were primary pre-service teachers it cannot be assumed that the 
additional study included mathematics. 

Factors Not Associated with Knowledge Differences 

None of the factors of university, full-or part-time study, anticipated graduation 
year, or on- and off-campus study modes was associated with differences in pre¬ 
service teacher performance on any of the scales. The only one of these that 
might be expected to impact teachers' knowledge would be anticipated 
graduation year, with those nearer to graduation having more knowledge than 
those further from the end of their course. This would amount to greater 
opportunity to learn which was associated with greater MCK and PCK for US, 
but not Singaporean, pre-service teachers in the TEDS-M study (Schmidt et al., 
2012; Wong et al., 2012). The courses studied by participants in this study varied 
in length from 1 to at least 4 years. A pre-service teacher planning to graduate in 
the next year, although near the end of his/her course, could be in the first, 
second, or fourth year of the program, with consequent implications for the 
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opportunities to learn that they had experienced. The inclusion of part-time 
students in the sample further complicates the interpretation of anticipated 
graduation year. To the extent that anticipated graduation year is a valid measure 
of the amount of mathematics education received, the apparent lack of impact on 
knowledge is of concern. More detailed information than was provided by the 
survey used in this study—about the nature, and placement within courses of 
mathematics content and mathematics curriculum units, along with details of by 
whom and how they are delivered—is needed to unpack whether, and to what 
extent, pre-service teacher mathematics education influences teacher knowledge. 

Scales for which There Were No Between-Group Differences 

There were no between-group differences for either Beliefs or PCK in spite of the 
diversity of the participants' backgrounds and of the courses and institutions in 
which they were enrolled. As discussed in the previous section, more detailed 
information about teacher education programs that provides a more nuanced 
view of pre-service teachers' experiences than anticipated graduation year are 
needed to better evaluate the impact of teacher education on these aspects. 

The difficulty of changing beliefs has been well established (e.g., Ambrose, 
2004; Lerman, 1997). Nevertheless, an aim of at least some teacher education 
programs is to influence the beliefs of pre-service teachers towards those 
considered helpful in terms of underpinning student-centred teaching practice. 
Several studies have reported success (e.g., Ambrose, 2004; Beswick, 2006) but 
most, including Ambrose (2004) and Beswick (2006), have been accompanied by 
scepticism about the longevity of apparent change. The results of this study show 
no impact of time in teacher education courses, as measured (albeit imperfectly) 
by anticipated graduation year, on pre-service teachers' beliefs. One possible 
explanation of this result is that the pre-service teachers had not reflected on the 
nature of mathematics in order to construct beliefs about it that could provide a 
coherent basis for their views about teaching and learning the subject. Although 
happy to adopt the rhetoric of student-centred teaching, they may not, as 
suggested by Beswick and Callingham (2011b), have gone beyond the emotional 
attractiveness of such statements to consider their implications. 

The absence of an association between performance on PCK items and 
confidence, even though there was an association between confidence and MCK, 
suggests that pre-service teachers may be aware of, and concerned about, the 
need to know mathematics content but less aware of the extent of their PCK or 
of its importance. It could also reflect greater familiarity with mathematics 
content tests compared to tests of their PCK. 

PCK remains poorly understood and difficult to operationalise and 
measure. These facts underpinned the difficulty of devising items to measure it 
that were of an appropriate level of difficulty. In contrast to MCK, there is little 
agreement or confidence about measures of PCK (Chick, 2011). These difficulties, 
along with the facts that pre-service teachers are unlikely to have considered the 
concept of PCK prior to their teacher education course, and that it may or may 
not be explicitly discussed even in that context, explain why pre-service teachers 
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could be expected to be less aware of their ability in terms of PCK than in relation 
to MCK. Although PCK has been a topic of interest to mathematics educators 
since Shulman (1987) introduced it, much of the work in the area to date has 
focussed on conceptualising the construct (e.g.. Chick et al., 2006; Hill et al., 2008) 
as a necessary pre-cursor to measuring it, and this work is far from complete. The 
items developed and used in this study contribute to this work and the findings 
emphasise the importance of continued research on PCK—its meaning, 
measurement, and development. 

The lack of any relation to PCK according to either prior education level or 
highest mathematics or statistics studied is consistent with PCK being uniquely 
addressed by education courses such that differences in entering knowledge and 
experience are immaterial. Nevertheless, Baumert et al.'s (2010) reminder of the 
importance of MCK as the basis of PCK suggests that MCK is also an important 
concern of teacher education. In addition, the data suggest that neither how nor 
where (mode, full or part time, course type, university) education study is 
undertaken make a difference to the level of PCK acquired. The findings suggest 
that efforts to improve the quality of graduating teachers in relation to the kind 
of knowledge that is central and unique to the act of teaching (as well as most 
connected with student outcomes as asserted by Baumert et al.) should not be 
overly-focussed on input variables or on gross structural aspects of courses and 
study modes. Rather, research addressing, in detail, the ways in which PCK 
manifests in teaching; the thinking, skills, and mathematics knowledge that 
underpin it; and how its development among pre-service teachers can be 
enhanced is very much needed. 


Conclusion 

This study is timely in relation to national agendas around school curricula 
(ACARA, 2012), and national teacher and teacher education program standards 
(AITSL, 2011a; 2011b). The survey results reported provide the beginnings of an 
evidence base in Australia for the ongoing improvement of the quality of 
mathematics teacher education and hence of graduates. 

Of particular relevance are the relatively few associations between the 
background variables and aspects of teacher knowledge that were measured, 
with only MCK and the overall measure. Teacher Knowledge, differing 
according to any. Unsurprisingly, the more mathematical or general education 
that the pre-service teachers had experienced—essentially opportunities to learn 
(Tatto & Senk, 2011)—the better their performance on both measures. Equally 
unsurprisingly, greater confidence to teach mathematics at the level they would 
be qualified to teach it was also associated with more MCK and Teacher 
Knowledge. PCK performance, which, in contrast to MCK, has been shown to 
predict student outcomes, showed no difference in relation to any of the 
grouping variables. There were, similarly, no differences in relation to beliefs. 
Whereas influencing beliefs is recognised as difficult (Ambrose, 2004) there is an 
expectation that PCK can be learned. Nevertheless, both constructs may not be 
well operationalised by multiple choice survey items such as used in this study 
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(Chick, 2011). The need to test theories using large-scale data collection, however, 
means that the effort to develop and validate readily scored survey instruments 
cannot be abandoned. The relative difficulty of the PCK items and ease of the 
beliefs items for the participants in this study suggest a need to devise PCK and 
beliefs items that better match the ability of primary pre-service teachers; that is, 
easier PCK items and more difficult beliefs items. The difficulty of agreeing on 
the scoring of PCK items (Chick, 2011) is consistent with Beswick et al.'s (2012) 
conception of teacher knowledge as including beliefs in that it highlights the role 
that the researchers' beliefs played in judging the relative merits of teaching 
responses in terms of the PCK that they evidenced. Items that present scenarios 
that include more of the detail and complexity of the contexts in which PCK is 
enacted and which offer options that also imply particular pedagogic stances, or 
beliefs about mathematics teaching and learning, may be a way forward. 

The absence of associations of any of the knowledge measures with the 
structural aspects of teacher education programs or even with anticipated 
graduation year could be an artefact of the measures or of the lack of precision of 
anticipated graduation year as a proxy for the opportunity to learn that 
respondents had experienced. Interview data from the larger study may provide 
important insights into validity of the measures and the subtleties of the impacts 
of teacher education programs. Whether or not this proves to be the case, the 
current policy environment makes evidence-based teacher education a priority. 
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